# heatmap to approximate which relationships are more convenient to review.
f,ax = plt.subplots(figsize = (34,20))
sns.heatmap(df.corr(),\
annot = True,\
linewidths=.5,\
fmt = '.1f',\
ax = ax,\
cbar_kws={'label': 'heat index'})\
.set_title('correlation dataset')
plt.xlabel("Values on X axis")
plt.ylabel('Values on Y axis')
We review some characteristics of the dataset¶
#
print("shape: {}\n".format(df_final.shape))
df_final.info()
df_final.describe()
What is the structure of your dataset?¶
Here we find 113,937 loans and we will use 18 columns, most of the data is numeric.
Data dictionary¶
| Column | Description |
|---|---|
| ListingNumber | The number that uniquely identifies the listing to the public as displayed on the website. |
| ListingCategory (numeric) | The category of the listing that the borrower selected when posting their listing: 0 - Not Available 1 - Debt Consolidation 2 - Home Improvement 3 - Business 4 - Personal Loan 5 - Student Use 6 - Auto 7- Other 8 - Baby&Adoption 9 - Boat 10 - Cosmetic Procedure 11 - Engagement Ring 12 - Green Loans 13 - Household Expenses 14 - Large Purchases 15 - Medical/Dental 16 - Motorcycle 17 - RV 18 - Taxes 19 - Vacation 20 - Wedding Loans |
| Term | The length of the loan expressed in months. |
| BorrowerAPR | The Borrower's Annual Percentage Rate (APR) for the loan. |
| BorrowerRate | The Borrower's interest rate for this loan. |
| LenderYield | The Lender yield on the loan. Lender yield is equal to the interest rate on the loan less the servicing fee. |
| EstimatedEffectiveYield | Effective yield is equal to the borrower interest rate (i) minus the servicing fee rate, (ii) minus estimated uncollected interest on charge-offs, (iii) plus estimated collected late fees. Applicable for loans originated after July 2009. |
| EstimatedLoss | Estimated loss is the estimated principal loss on charge-offs. Applicable for loans originated after July 2009. |
| EstimatedReturn | The estimated return assigned to the listing at the time it was created. Estimated return is the difference between the Estimated Effective Yield and the Estimated Loss Rate. Applicable for loans originated after July 2009 |
| ProsperRating (numeric) | The Prosper Rating assigned at the time the listing was created: 0 - N/A 1 - HR 2 - E 3 - D 4 - C 5 - B 6 - A 7 - AA Applicable for loans originated after July 2009. |
| ProsperScore | A custom risk score built using historical Prosper data. The score ranges from 1-10, with 10 being the best, or lowest risk score. Applicable for loans originated after July 2009. |
| CurrentCreditLines | Number of current credit lines at the time the credit profile was pulled. |
| OpenCreditLines | Number of open credit lines at the time the credit profile was pulled. |
| TotalCreditLinespast7years | Number of credit lines in the past seven years at the time the credit profile was pulled. |
| OpenRevolvingAccounts | Number of open revolving accounts at the time the credit profile was pulled. |
| OpenRevolvingMonthlyPayment | Monthly payment on revolving accounts at the time the credit profile was pulled |
| TotalTrades | Number of trade lines ever opened at the time the credit profile was pulled. |
| Investors | The number of investors that funded the loan. |
# let's do a general review of the variables
df_final.hist(figsize=(20,12));
# Prosper Rating hist
bin_edges = np.arange(0, df_final['ProsperRating (numeric)'].max()+1, 1)
plt.figure(figsize=[12, 8])
plt.hist(data = df_final, x = 'ProsperRating (numeric)', bins = bin_edges)
plt.xticks([0, 1, 2, 3, 4, 5, 6, 7], ['N/A', 'HR', 'E', 'D', 'C', 'B', 'A', 'AA'])
plt.xlabel('Prosper Rating')
plt.ylabel('Quantity')
plt.title('Prosper Rating types', fontweight="bold")
plt.show()
We could say that the Prosper Rating of the applicants in most cases is on the rise, located in the most risky, where we observe a small decrease with respect to the pattern is type A
# Prosper Score hist
bin_edges = np.arange(0, df_final['ProsperScore'].max()+1, 1)
plt.figure(figsize=[12, 8])
plt.hist(data = df_final, x = 'ProsperScore', bins = bin_edges)
plt.xticks([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12'])
plt.xlabel('Prosper Score')
plt.ylabel('amount')
plt.title('Prosper Score', fontweight="bold")
plt.show()
We can the scores of the users are mostly between 5 and 9
# Estimated Loss hist
bin_edges = np.arange(0, df_final['EstimatedLoss'].max()+1, 1)
plt.figure(figsize=[12, 8])
plt.hist(data = df_final, x = 'EstimatedLoss')#, bins = bin_edges)
#plt.xticks([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12'])
plt.xlabel('EstimatedLoss')
plt.ylabel('amount')
plt.title('Estimated Loss', fontweight="bold")
plt.show()
We observe in this graph that the estimated losses are mostly quite low.
# Estimated Return hist
bin_edges = np.arange(0, df_final['EstimatedReturn'].max()+1, 1)
plt.figure(figsize=[12, 8])
plt.hist(data = df_final, x = 'EstimatedReturn')#, bins = bin_edges)
#plt.xticks([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12'])
plt.xlabel('EstimatedReturn')
plt.ylabel('amount')
plt.title('Estimated Return', fontweight="bold")
plt.show()
classically always the estimated return rate is higher than the loss rate
# Boxplot for estimated loss
plt.subplots(figsize=(18,10))
bp = plt.boxplot(df_final['EstimatedLoss'], vert=False, showmeans=True)
plt.title(('Estimated Loss boxplot'), fontweight="bold")
plt.xlabel('Estimated Loss')
There are still some outliers left, but let's see the median is a little lower than the mean, and the losses are found with a mean of 0.08
Bivariate Exploration¶
For this analysis we have the majority of variables that are related to each other, so we are going to make some plots to observe the relationships
# heatmap
f,ax = plt.subplots(figsize = (34,20))
sns.heatmap(df_final.corr(),\
annot = True,\
linewidths=.5,\
fmt = '.1f',\
ax = ax,\
cbar_kws={'label': 'heat index'})\
.set_title('correlation dataset')
plt.xlabel("Values on X axis")
plt.ylabel('Values on Y axis')
# Subplots for BorrowerAPR
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of BorrowerAPR's closest relationships")
columns = 3
rows = 2
column = ['BorrowerRate', 'LenderYield', 'EstimatedEffectiveYield', 'EstimatedLoss', 'EstimatedReturn']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'BorrowerAPR',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("BorrowerAPR vs {}".format(column[i-1]))
plt.show()
As we can see a strong positive relationship between BorrowerAPR and BorrowerRate / LenderYield, but with the other variables, even though a positive relationship is maintained, it becomes less strong.
# Subplots for BorrowerRate
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of BorrowerAPR's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'LenderYield', 'EstimatedEffectiveYield', 'EstimatedLoss', 'EstimatedReturn']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'BorrowerRate',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("BorrowerRate vs {}".format(column[i-1]))
plt.show()
With the variable BorrowerRate, the strongest relationship is LenderYield then BorrowerAPR, and hence the relationship is a bit more unstable, but always positive.
# Subplots for LenderYield
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of LenderYield's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'BorrowerRate', 'EstimatedEffectiveYield', 'EstimatedLoss', 'EstimatedReturn']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'LenderYield',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("LenderYield vs {}".format(column[i-1]))
plt.show()
The LenderYield variable has the same behavior as BorrowerRate, and hence its perfect relationship.
# Subplots for EstimatedEffectiveYield
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of EstimatedEffectiveYield's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'BorrowerRate', 'LenderYield', 'EstimatedLoss', 'EstimatedReturn']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'EstimatedEffectiveYield',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("EstimatedEffectiveYield vs {}".format(column[i-1]))
plt.show()
# Subplots for EstimatedLoss
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of EstimatedLoss's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'BorrowerRate', 'LenderYield', 'EstimatedEffectiveYield', 'EstimatedReturn']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'EstimatedLoss',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("EstimatedLoss vs {}".format(column[i-1]))
plt.show()
# Subplots for EstimatedReturn
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Scatterplot of EstimatedReturn's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'BorrowerRate', 'LenderYield', 'EstimatedEffectiveYield', 'EstimatedLoss']
for i in range(1, 6):
fig.add_subplot(rows, columns, i)
sns.scatterplot(x = 'EstimatedReturn',\
y = column[i-1],\
alpha = 0.5,\
data = df_final).set_title("EstimatedReturn vs {}".format(column[i-1]))
plt.show()
The following variables have positive relationships with the other variables but it is a bit uneven with respect to the first visualizations.
# Subplots for ProsperRating (numeric)
fig=plt.figure(figsize=(24, 12))
fig.suptitle("Barplot of ProsperRating's closest relationships")
columns = 3
rows = 2
column = ['BorrowerAPR', 'BorrowerRate', 'LenderYield', 'EstimatedEffectiveYield','EstimatedReturn', 'EstimatedLoss']
for i in range(1, 7):
fig.add_subplot(rows, columns, i)
sns.barplot(x = 'ProsperRating (numeric)',\
y = column[i-1],\
data = df_final).set_title("ProsperRating vs {}".format(column[i-1]))
plt.xticks([0, 1, 2, 3, 4, 5, 6], ['HR', 'E', 'D', 'C', 'B', 'A', 'AA'])
plt.show()
Here we see a negative relationship between ProsperRating vs BorrowerAPR / BorrowerRate / LenderYield / EstimatedEffectiveYield / EstimatedReturn / EstimatedLoss
# violin plot
plt.figure(figsize = [24, 10])
ax = sns.violinplot(data = df_final, x = 'ProsperRating (numeric)', y = 'ProsperScore')
plt.xticks([0, 1, 2, 3, 4, 5, 6], ['HR', 'E', 'D', 'C', 'B', 'A', 'AA'])
ax.set_title("ProsperRating (numeric) vs ProsperScore")
plt.show()
In this plot we can see how the higher the ProsperRating, the higher the ProsperScore, a positive relationship but represented in another way.
Some conclusions¶
As we can see in the graphs, there is a fairly close relationship between the borrower and estimated columns, this makes us know that the more one grows, the greater the other will be, which gives us greater clarity about the risks of these loans.
We also observe the interesting relationship between prosper rating and prosper score, where we observe that most of the users evaluated with high degrees are the safest to invest, and have a higher prosper score, which is good since it generates mutual security.
Multivariate Exploration¶
Here we will try to use the "year" column for the following graphs, and we will try to make some interesting plots about the loan.
# first we must go to prosper rating to letters
new_prosper_rating = {'ProsperRating (numeric)': {0:'N/A', 1:'HR', 2:'E', 3:'D', 4:'C', 5:'B', 6:'A', 7:'AA'}}
df_final.replace(new_prosper_rating, inplace = True)
df_final.head()
In these plots we mostly use the Prosper rating as the third variable with respect to relationships.
sns.lmplot(x='BorrowerAPR',height = 10, y='EstimatedLoss', hue='ProsperRating (numeric)',
fit_reg=True, data=df_final)
plt.show()
As we can see in the relationship between BorrowerAPR and EstimatedLoss, the prosper rating greatly influences, since the higher the rank, the lower the estimated loss, since they are the most reliable users for loans.
plt.figure(figsize = [24, 14])
sns.pointplot(data = df_final, x = 'TotalTrades', y = 'TotalCreditLinespast7years', hue = 'ProsperRating (numeric)',
palette = 'Blues', linestyles = '', dodge = 0.4)
plt.xticks(rotation='vertical');
plt.show()
Another interesting graph about the relationship between the total trade with that of the past 7 years, we can see that it is constant and as the highest prosper ratings mostly have fewer trades
g = sns.FacetGrid(data = df_final, height = 10, col = 'year', col_wrap = 5, hue = 'ProsperRating (numeric)')
g.map(sns.countplot, "ProsperScore")
leg = plt.legend(loc = 'upper right', title = 'ProsperRating (numeric)')
plt.show()
Looking at the passing of the years, it is observed that in the more recent years the prosper rating and score relationship is stronger, and with more consistency, since the higher the rank, the higher the score.
sns.relplot(x="LenderYield",height = 20, y="EstimatedReturn", hue="ProsperRating (numeric)",
kind="scatter", data=df_final);
This last graph is another example of how the higher the prosper rating, the lower the loss and the credit payment.
Some conclusions¶
As we observe in the plots, the prosper rating and score are fundamental to deduce the increase or decrease of the other loan relationships that have to do with the borrower and estimates of return or loss, and even though the column of years is not decisive, it can be see with interest the behavior of the other variables.
Limitations of Project:¶
There are few qualitative relationships, beyond the prosper rating, since there are still other similar columns that do not keep much correlation with the other variables.
I think that the information provided about the loans is not very enriching, and I tried to understand a little from the outside but I do not know if it captures well the possible concerns to be raised.
Too many variables, I took the ones that I saw that had a better relationship between them, but maybe someone more expert in loans could make a more successful decision.